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SUMMARY 

This  keynote  presents  a  framework  for  incorporating  cognitive  automation  into  a  work  system,  not  only  as 
part  of  the  operation-assisting  means  but  also  in  terms  of  cognitive  cooperation  of  a  human-machine  team 
operating  the  work  system  and  thereby  taking  the  high-end  decisions  of  the  work  process.  This  framework  is 
based  on  the  experience  with  successful  prototype  development  of  cognitive  assistant  systems  in  the  flight 
systems  and  road  vehicle  domain.  A  theoretical  framework  as  shown  is  important  in  order  to  be  able  to  assess 
work  system  designs  with  regard  to  productivity,  i.e.  effectiveness  and  safety. 

1.0  INTRODUCTION 

Man-machine  cognitive  cooperation  has  become  reality.  There  are  a  great  number  of  prototype  systems, 
which  have  demonstrated  the  capabilities  of  cognitive  cooperation.  Some  references  are  given  for  examples 
concerning  the  prominent  pilot  work  site  in  an  aircraft  cockpit  only  [Onken  et  al.,  89;  Lizza  et  al.,  91,92; 
Kopf  et  al.,  93;  Champigneux,  95;  Gerlach  et  al.,  95;  Funk  et  al.,  98;  Miller  et  al.,  1999;  Reising  et  al.,  99; 
etc.],  however,  there  are  many  other  fields  of  application.  These  systems  have  got  artificial  cognitive 
capabilities  and  thereby  allow  teaming  with  human  operators  as  well  as  with  other  artificial  cognitive  systems 
in  order  to  actively  assist  the  operator  in  a  work  process,  mainly  regarding  situation  awareness  and  effective 
actions  with  respect  to  the  situational  context.  The  most  salient  feature  of  these  systems  is  that  they  have  a 
great  amount  of  knowledge  in  common  with  the  human  operator  to  be  able  to  carry  out  the  work  tasks, 
if  necessary,  even  on  their  own.  Most  important,  they  have  got  explicit  knowledge  of  the  prime  work  system 
objectives,  the  key  knowledge  for  a  cooperative  system  to  catch  up  with  the  intentions  of  the  team  mate, 
for  instance  the  human  operator,  and  to  warrant  sensible  interaction  as  a  team  mate.  Since  prototype  systems 
have  demonstrated  that  cognitive  cooperation  is  technically  feasible,  the  following  chapters  will  not  dwell  on 
the  technical  details  of  artificial  cognitive  systems.  The  focus  will  rather  be  on  the  underlying  fundamental 
ideas  about  artificial  cognitive  systems  and  intrinsic  potentials  for  cognitive  cooperation. 

Flence,  this  keynote  will  dwell  on  the  main  properties  of  artificial  cognitive  systems,  how  these  properties  can 
be  used  to  improve  work  system  capabilities  and  to  make  work  processes  more  productive  and  more  efficient. 
It  will  point  out  that  automation  in  the  new  setting  of  cognitive  automation  will  bring  about  human-machine 
cognitive  cooperation  in  a  way  similar  to  human-human  cognitive  cooperation  [Hoc  et  al,  1995]  and  that  it  can 
be  exploited  for  more  effective  coordination  between  team-mates,  humans  as  well  as  machines.  Approaching 
this  topic  from  the  system  engineer’s  point  of  view,  the  structure  of  the  work  system  and  the  possible  ways  to 
incorporate  artificial  cognition  in  the  work  system  have  to  be  discussed  first. 
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2.0  HOW  DOES  ARTIFICIAL  COGNITION  EFFECT  THE  WORK  SYSTEM? 

2,1  The  Concept  of  Work  System 

Taking  the  system  engineer’s  point  of  view  to  find  out  how  artificial  cognition  can  effect  human-machine 
interaction,  one  should  start  with  the  concept  of  work  system.  The  work  system  is  traditionally  structured  as 
shown  in  figure  1.  There  are  three  main  work  system  elements:  Operator,  work  object  and  operator-assisting 
means  [REF A,  84].  The  way  the  work  system  functions  forms  the  work  process.  The  work  process  operates 
under  certain  ambient  conditions  on  a  given  work  task,  i.e.  to  achieve  a  certain  time-dependent  work  result  in 
terms  of  a  transformation  of  the  work  object.  Thereby,  the  work  process  makes  use  of  certain  resources 
(information,  material,  energy).  A  work  system  can  only  be  set  up  based  on  a  high  level  work  task  which 
stands  for  itself,  i.e.  being  independently  carried  out  from  any  other  work  task,  like  building  a  house  or  flying 
an  aircraft  from  one  certain  location  to  another  etc.  Thus,  performing  a  subtask  like  holding  speed  constant 
during  a  certain  flight  segment,  which  is  obviously  only  part  of  a  higher  level  work  task,  does  not  form  a 
separate  work  system.  The  effectiveness  of  the  work  system  is  determined  by  the  following  elements: 
Operator,  operation-assisting  means,  environmental  conditions  and  resources.  The  operator  is  the  high  end 
decision  element  of  the  work  system.  It  is  only  this  element  which  determines  and  supervises  within  the  work 
system  what  will  happen  with  the  work  object  in  order  to  accomplish  the  work  task.  With  respect  to  the  work 
system  concerned,  this  element  operates  autonomously.  While  the  human  operator  and  the  environmental 
conditions  cannot  be  changed  beyond  certain  narrow  limits,  and  since  the  aspect  of  resource  consumption  is 
more  or  less  an  economical  issue,  which  is  important,  though,  but  not  so  relevant  for  the  ergonomic  design, 
it  makes  sense  that  today  ergonomic  scientists  increasingly  concentrate  on  the  improvement  of  the  operation- 
assisting  means,  i.e.  systematically  and  methodically  exploiting  technology  for  the  sake  of  a  human-centred 
and  efficient  work  system.  In  the  past,  computerized  operation-assisting  means  often  have  been  implemented 
without  use  of  well-founded  methods  and  theoretical  frameworks.  This  is  the  reason  why  the  potential  of 
automation  for  the  improvement  of  the  operation-assisting  means  has  been  underestimated  and  misunderstood 
for  a  long  time,  despite  increasing  its  use  dramatically,  too  often  not  to  the  delight  of  the  ergonomic  scientists. 
More  recently,  there  are  also  thoughts  about  having  the  high  end  decision  element  to  operate  the  work  system 
being  extended,  not  only  by  humans  but  also  by  artificial  cognition,  which  will  be  taken  up  later  on  in  this 
article. 


Figure  1:  Concept  of  Work  System. 
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2,2  The  Conception  of  an  Artificial  Cognitive  Unit  (ACU) 

The  conception  (model)  of  an  Artificial  Cognitive  Unit  (ACU)  will  be  briefly  described  in  the  following. 
The  ACU  has  got  cognitive  core  capabilities  which  are  of  great  similarity  to  those  of  humans  regarding 
rational  reasoning  and  decision  making  in  order  to  recognize/identify  the  encountered  situation  and  to  make 
action  plans  in  order  to  react  properly  subject  to  given  objectives.  An  ACU,  though,  is  not  to  mimic  the 
processes  of  the  human  brain  in  detail.  It  rather  should  be  able  to  generate  outputs  which  could  be  generated 
as  well  by  human  rational  thinking  or  which,  at  least,  are  intelligible  to  humans. 

Figure  2  shows  the  principal  structure  of  an  ACU.  The  central  component  is  the  “body”,  the  oval  core  of  the 
ACU  in  figure  2,  which  hosts  all  data  used  and  produced  by  the  artificial  cognitive  process.  The  inner  part  of 
the  body,  slightly  darker,  contains  the  “a  priori  knowledge”  which  is  fed  into  the  ACU  (or  learned),  before  the 
process  starts.  This  “a  priori  knowledge”  is  the  origin  of  application-dependent  behaviour  of  the  ACU. 
The  outer  part  of  the  body  contains  the  situational  “knowledge”  which  is  created  during  runtime.  This  specific 
kind  of  dynamically  generated  and  refurbished  knowledge  results  from  the  ACU-subprocesses, 
the  “transformators”  and  is  called  the  “cognitive  yield”. 


The  transformators  are  located  around  the  body.  They  have  access  to  all  a  priori  knowledge  and  cognitive 
yield  in  the  body  and  write  their  results  into  designated  areas  of  the  body  (at  the  arrowhead). 
The  transformators  represent  functions  according  to  those  of  the  recognition- act  cycle: 

•  Interpretation  (of  the  situation),  mainly  based  on  the  input  data  accessible  from  sensors  and 
communication  interfaces  outside  the  ACU  as  part  of  the  environment  and  a  priori  knowledge  about 
environment  models', 

•  Goal  determination  (including  determination  of  conflicts  and  opportunities),  mainly  based  on  the 
belief  as  output  of  the  interpretation  transformator  and  a  priori  desires. 


The  a  priori  knowledge  of  environmental  models  might  be  extended  -  if  wanted  -  also  by  introspective  models  of  the  ACU  itself. 


RTO-MP-088 


KN5-3 


Cognitive  Cooperation  for  the  Sake  of  the  Human-Machine  Team  Effectiveness 


ORGAI^ZATtOM 


•  Planning,  mainly  based  on  the  determined  goals  and  strategy  models  and 

•  Plan  realisation,  mainly  based  on  the  plan  how  to  proeeed  and  a  priori  knowledge  about  adequate 
instruetions  for  the  execution  units  available  to  the  ACU  as  part  of  its  environment. 

The  environment  represents  the  world  in  real  the  ACU  is  interacting  with.  This  includes  the  other  components 
of  the  work  system  the  ACU  belongs  to,  including  human  operators  and  environmental  objects,  but  also  other 
work  systems  of  relevance. 

As  was  already  alluded  to,  also  the  learning  capability  can  be  represented  in  such  a  unit,  if  wanted.  It  can  be 
imagined  in  terms  of  a  second  layer,  fed  by  the  belief  and  supplementing  the  a  priori  knowledge.  This  is  not 
depicted  in  figure  1  and  will  not  be  further  discussed  at  this  point,  though. 

The  ACU  represents  a  computer  software,  exploiting  the  computer  technology  available  in  order  to  achieve  a 
cognitive  performance  level  as  high  as  possible.  This  includes  to  incorporate  all  relevant  knowledge  available 
about  human  cognition,  but  there  is  not  the  principal  intention  to  develop  an  ACU  in  order  to  mimic  the 
physiology  of  the  human  brain  as  closely  as  possible.  This  would  be  too  specific  and  limiting  regarding  the 
ACU  potentials  in  general.  ACU  software  is  available  at  this  point  of  time,  even  in  terms  of  an  application- 
independent  software  [Putzer,  01].  Although  this  already  provides  great  potentials  for  improvement  of  work 
systems,  in  particular  for  more  complex  systems,  there  also  is,  on  the  other  hand,  still  much  room  for  further 
performance  improvement  of  the  ACU. 

2,3  The  Two  Faces  of  Automation 

In  general,  automation  is  a  technical  resource  with  the  capability  to  carry  out  on  its  own  tasks  of  a  work 
process  as  determined  by  the  work  system  designer  or  the  operator.  Hence,  there  is  the  potential  that 
automation  can  help  to  avoid  excessive  load  on  the  operator,  i.e.  to  keep  the  load  on  the  operator  at  an 
acceptable  level.  Otherwise,  all  tasks  had  to  be  carried  out  by  the  operator  alone. 

Automation,  as  it  is  realistically  feasible  today,  including  artificial  cognition,  can  appear  in  many  ways  in  the 
work  system.  Thereby,  it  might  show  mainly  two  faces,  each  of  them  with  significantly  different 
characteristics  [Onken,  01].  In  other  words,  we  know  of  two  significantly  different  levels  in  automation 
quality  which  are  possible  in  the  meantime:  One  of  them  is  what  we  are  used  to  as  the  conventional  approach 
(we  call  it  “conventional  automation”),  as  opposed  to  the  “cognitive  automation”  with  characteristics  which 
we  are  familiar  with  as  those  of  human  team  mates. 

Level  1  (Conventional  Automation) 

Conventional  automation,  whether  activated  automatically  according  to  design  specifications  or  activated 
manually  by  the  operator,  is  predominantly  focused  on  subtasks  and  accordingly  on  sub-goals  of  the  work 
process.  The  superior  prime  goals  and  pertinent  tasks  are  not  known  by  the  systems  for  automation  of 
subtasks.  Consequently,  these  systems  work  on  the  basis  of  observing  only  a  small  portion  of  what  is 
of  relevance  in  a  given  situation.  For  example,  if  the  autopilot  is  activated  by  the  pilot  for  the  subtask 
“altitude  hold”,  it  is  doing  its  best  to  comply  with  this  assigned  function,  even  if  a  high  mountain  is  in  its  way. 
The  top-level  safety  goal  of  avoiding  a  crash  into  the  mountain  is  exclusively  in  the  realm  of  the  pilot’s 
responsibilities.  The  autopilot  doesn’t  know  of  it  and  will  not  care. 

This  conventional  concept  of  having  automation  functions  in  the  work  process  like  specialists  who  are  good 
for  a  particular  task  but  who  lack  perspective  otherwise,  was  very  convenient  as  long  as  only  simple 
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automated  functions  were  used  for  simple  tasks.  It  was  of  no  harm  that  the  responsibility  load  not  to  violate 
the  top-level  objectives  of  the  work  process  was  exclusively  on  the  operator’s  shoulders.  However,  in  work 
systems  for  very  complex  tasks  like  those,  for  instance,  for  carrying  out  a  demanding  fighter  mission, 
increasingly  complex  “operator-assisting”  automation  means  have  to  accommodated,  correspondingly. 
With  increasing  automation  complexity,  the  conventional  concept  inevitably  will  lead  to  overload  of  the 
operator  at  some  point,  resulting  in  a  dangerous  loss  of  performance.  The  operator  might  be  unaware  of 
discrepancies  between  subtask  activities  of  automated  functions  and  the  prime  goal  necessities  or  of 
insufficient  adaptiveness  on  his  side.  This  is  just  the  opposite  of  what  was  intended  by  the  introduction  of 
automation  and  is  known  as  automation  brittleness,  opacity,  literalism  and  clumsiness  [Billings,  97].  Figure  3 
is  showing  a  simple  diagram,  illustrating  that  for  the  pilot’s  work  site.  In  fact,  conventional  automation 
reduces  the  demand  on  the  pilot’s  resources  on  the  average  most  of  the  time,  but  might  also  generate  excessive 
load  -  at  rare  occasions,  though  -  which  would  not  have  happened  without  automation.  Many  accidents  can  be 
accounted  to  that  phenomenon. 

Demand  on  Pilot  Resources 


Level  2  (Cognitive  Automation) 

As  opposed  to  conventional  automation,  cognitive  automation  works  on  the  basis  of  comprehensive 
knowledge  about  work  process  objectives  and  goals  on  all  goal  hierarchy  levels,  pertinent  task  options  and 
necessary  data  describing  the  current  situation  in  the  work  process.  Therefore,  cognitive  automation  is  prime- 
goal-oriented.  In  summary,  it  stands  for  artificial  capabilities: 

•  to  independently  assess  and  keep  ready  necessary  situation-relevant  information  about  the  objectives 
the  human  operator  is  pursuing,  about  his  intents  and  activities,  about  the  work  object  and  tools, 
and  about  the  relevant  process  environment; 

•  to  understand  the  situation  by  independently  interpreting  it  in  the  light  of  the  objectives; 

•  to  distinguish  between  important  and  unimportant  information,  urgent  and  less  urgently  needed 
actions; 

•  to  know  which  information  the  operator  needs; 
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•  to  support  necessary  re-planning  and  decision  making;  and 

•  to  initiate  human-like  communication  with  the  operator,  thereby  actively  taking  care  that  the 
operator’s  situation  awareness  is  evened  up  with  what  is  detected  as  conflicts  or  opportunities  by  the 
systems,  not  to  leave  him  alone  with  presentations  which  do  not  care  about  what  he  has  understood 
about  the  situation  and  what  he  actually  perceives  or  does  not  perceive. 

Making  use  of  these  capabilities  in  terms  of  operation-assisting  means  in  the  work  system,  it  has  no  longer  to 
be  the  exclusive  task  of  the  operator  to  monitor  the  process  subject  to  the  prime  work  system  objectives. 
ACUs  working  on  prime  work  system  objectives  identical  to  those  of  the  human  operator,  that  is  something 
the  designers  of  work  systems  have  yet  to  get  used  to  [Vicente,  99].  Then,  a  “cognitive”  autopilot  will  see  the 
mountain  in  front,  will  know  that  to  proceed  stubbornly  with  altitude  hold  will  end  in  disaster  and  it  will  look 
for  a  way  around.  Consequently,  figure  1  can  be  drawn  in  more  detail  making  explicit  the  possible 
incorporation  of  ACUs  as  part  of  the  operator  assisting  means  in  the  work  system  (see  figure  4). 


Environmental  Conditions 


ACU  =  Artificial  Cognitive  Unit 

Figure  4:  Work  System  with  Cognitive  Automation  (ACUs)  as  Part  of  the  Operation-Assisting  Means. 

This  does  not  mean  that  cognitive  automation  is  to  be  used  exclusively.  Level  1  automation  will  be 
incorporated,  too,  as  depicted,  and  is  still  very  useful,  indeed,  for  low  complexity  functions,  as  was  proven 
oftentimes.  Cognitive  automation  on  the  ACU  level,  however,  is  the  basis  for  man-machine  cooperation  in  its 
real  sense.  Figure  5  summarizes  this  discussion  in  terms  of  possible  productivity  gains.  One  new  but  not  too 
difficult  design  question  is  now,  where  to  work  with  conventional  automation  and  where  to  use  cognitive 
automation. 
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Productivity  =  f(Safety,  profitability,  etc.) 


Figure  5:  Effect  of  Conventional  and  Cognitive  Automation  on  Productivity  and  Safety. 


3.0  COGNITIVE  COOPERATION  -  A  BY-PRODUCT  OF  COGNITIVE 
AUTOMATION 

As  opposed  to  mere  interaction,  cooperation  has  particular  main  characteristics.  Cooperating  units  in  a  work 
system  (see  figure  6)  are: 

•  commonly  having  access  to  all  information  available  through  sensors,  communication  and  databases; 

•  appearing  as  humans  and  as  ACUs  as  part  of  the  work  system^; 

•  capable  for  autonomy  but  not  necessarily  working  autonomously; 

•  in  general,  capable  to  adopt  the  roles  (i.e.  grouping  of  coherent  functions  allocated  to  separate 
cognitive  units)  of  other  cooperating  units  in  the  course  of  the  work  process,  because  all  are  working 
on  common  prime  work  system  objectives; 

•  working  rather  independently  subject  to  common  work  prime  objectives,  thereby  possibly  starting 
independent  initiatives,  usually  in  terms  of  communication  to  even  up  in  beliefs  about  the  situation, 
goals  and  decisions  and  to  coordinate  roles  and  authority  levels. 

This  means  that  cooperating  units  can  appear  as  assisting  means  (level  2  automation  ACUs)  in  the  work 
system  at  a  rather  low  decision  authority  level,  but  also  as  ACUs  and  human  operators  in  the  operator/ACU 
team  (see  figure  6)  at  the  high-end  authority  level  for  decisions  in  the  work  system,  which  was,  so  far, 
occupied  by  the  human  operator  alone.  Basically,  each  of  the  cooperating  units  can  (have  not  to,  though) 
carry  out  all  tasks,  which  might  come  up  in  the  course  of  a  work  process,  but  depending  on  the  role  and  the 
association  with  either  the  operator/ACU  team  or  the  assisting  means,  tasks  will  be  partitioned  between  them. 


2 


In  general,  also  other  biological  creatures  could  appear  as  cooperating  units  like,  for  instance,  the  horse,  dragging  the  carriage  and 
still  taking  care  of  avoiding  disaster,  when  the  coachman  has  gone  asleep  (horse  mode). 
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ACU  =  Artificial  Cognitive  Unit 

Figure  6:  Work  System  with  Cognitive  Cooperation  (Operators-ACUs). 


The  cooperating  units  in  the  operator/ACU  team  are  normally  cooperating  under  certain  complementary  roles 
within  the  work  process.  All  team  members  are  entitled  for  full  authority  (highest  in  command  line)  within  the 
respective  role  to  effect  the  work  object  (i.e.  autonomy).  Therefore,  autonomy,  on  principle  associated  with 
the  operation  of  a  particular  work  system,  is  distributed  between  cooperating  operator/ACU  team  members  by 
roles.  A  typical  example  for  this  kind  of  cooperation  in  a  human  team  is  the  cooperation  of  the  cockpit  crew, 
for  instance  the  TORNADO  crew  with  a  pilot  and  the  Weapon  Systems  Officer.  On  the  other  hand, 
the  operation  of  unmanned  systems,  for  instance,  needs  that  kind  of  work  system  setup  where  ACUs  as  part  of 
the  operator/ACU  team  have  to  take  over  decision  tasks  at  the  high-end  decision  level. 

Another  consequence  using  cooperating  cognitive  units  in  the  work  system  is  the  potential  that  subtasks  which 
do  not  end  up  in  a  direct  output  to  effectors  on  the  work  object,  might  be  taken  up  in  parallel  by  one  or  more 
other  cooperating  units,  what  is  a  well  known  fact  in  human  teams.  This  potential  might  be  effectively  used  by 
both  ACUs  as  assisting  means  and  ACUs  working  in  the  operator/ACU  team  at  the  high-end  authority  level. 
In  turn,  this  leads  to  the  other  potential  that  cooperating  units  are  not  bound  to  a  firm  allocation  of  tasks. 

Conflicts,  in  particular  decision  conflicts  between  cooperating  units  in  different  roles,  but  also  when  working 
in  parallel,  are  always  possible.  The  roles  of  cooperating  units  might  be  overlapping  in  certain  ways  and 
working  in  parallel  under  dissimilar  strategies  in  some  ways  is  just  so  important,  because  thereby 
misinterpretations  and  system  errors  can  be  made  evident.  Conflicts  are  to  be  avoided  by  an  authority 
hierarchy,  which,  in  the  ideal  case  of  high  flexibility,  might  be  subject  to  change.  Certain  circumstances  might 
cause  the  need  for  that.  For  the  case  of  human  teams  like  the  pilot  crew  in  a  transport  aircraft,  this  is  well 
known  under  the  term  “crew  coordination”  which  is  to  open  up  some  flexibility  in  the  authority  allocation  to 
team  members  in  the  course  of  the  work  process.  That  can  also  be  exploited  in  a  mixed  team  of  human 
operators  and  ACUs.  Usually,  the  human  operator’s  authority  level  is  above  the  authority  of  ACUs,  but  this 
has  to  be  carefully  investigated  for  the  particular  work  system  case  being  considered.  The  principal  of  higher 
authority  level  on  the  human  operator’s  side  is  not  necessarily  the  best  solution  in  all  cases.  Therefore, 
flexibility  of  authority  allocation  is  of  great  interest,  but  not  easy  to  implement. 

Referring  to  two  examples  of  prototype  systems  being  referenced  earlier,  the  “Cogpit”  project  in  the  UK 
[Taylor,  01]  and  the  CAMA  project  in  Germany  [Lenz,  00;  Frey  et  ah,  01]  came  already  rather  close  to  that 
kind  of  objective  for  team  coordination.  CAMA  (Crew  Assistant  for  Military  Aircraft)  is  an  assistant  for  the 
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crew  of  military  transport  aircraft,  developed  and  flight  tested  under  contract  of  the  German  DoD, 
and  “Cogpif  ’  (Cognitive  Cockpit),  is  a  demonstration  prototype  system  of  cognitive  assistance  for  the  fighter 
pilot,  developed  under  contract  of  the  British  DoD.  In  particular,  the  Pilot  Authorization  of  Control  Tasks 
(PACT)  framework  in  the  “cogpit”  system,  providing  the  necessary  and  sufficient  levels  of  authority  for  the 
task  automation  manager  system  is  addressing  this  objective  with  emphasis.  The  PACT  system  has  got  three 
primary  automation  modes,  quoting  from  [Taylor,  01], 

“  -  namely,  fully  automatic,  assisted  or  pilot  commanded  -  with  a  further  four  secondary  levels  nested 
within  the  semi-automatic,  assisted  mode,  which  can  be  changed  adaptively  or  by  pilot  command. 

The  PACT  system  uses  military  terminology  for  categories  of  support  for  Army  land  forces  military 
operations  (At  Cali,  Advisory,  In  Support,  Direct  Support)  to  afford  usability  and  compatibility  with 
military  user  cognitive  schemata  and  models.  It  provides  realistic  operational  relationships  for  a 
logical,  practical  set  of  levels  of  automation,  with  progressive  operator/piiot  authority  and  computer 
autonomy  supporting  situation  assessment,  decision  making  and  action.  Mission  functions  and  tasks, 
at  different  levels  of  abstraction  allocated  individually  or  grouped  in  related  scripts  or  plays,  can  be  set 
to  these  levels  in  a  number  of  ways: 

•  Pre-set  operator  preferred  defaults, 

•  Operator  selection  during  pre-flight  planning, 

•  Changed  by  the  operator  during  in-flight  re-planning,  probably  using  Direct  Voice 
Input  commands, 

•  Automatically  changed  according  to  operator  agreed,  context-sensitive  adaptive  rules.  ” 


Table  1 :  PACT  System  for  Pilot  Authorisation  of  Control  of  Tasks  [Taylor,  01] 


Primary 

Modes 

Levels 

Operational 

Relationship 

Computer 

Autonomy 

Pilot 

Authority 

Adaptation 

Information  on 
performance 

AUTOMATIC 

Automatic 

Full 

Interrupt 

Computer 
monitored  by 
pilot 

On/off 

Failure  warnings. 
Performance  only 
if  required. 

ASSISTED 

4 

Direct 

Support 

Advised  action 
unless  revoked 

Revoking 

action 

Computer 
backed  up  by 
pilot 

Feedback  on 
action.  Alerts  and 
warnings  on 
failure  of  action. 

3 

In  Support 

Advice,  and  if 

authorised, 

action 

Acceptance  of 
advice 

and  authorising 
action 

Pilot  backed  up 
by  the 
computer 

Feed-forward 
advice  and  feed¬ 
back  on  action. 
Alerts  and  warn¬ 
ings  on  failure  of 
authorised  action. 

2 

Advisory 

Advice 

Acceptance  of 
advice 

Pilot  assisted 
by  computer 

Feed-forward 

advice 

1 

At  Call 

Advice  only  if 
requested. 

Full 

Pilot,  assisted 
by  computer 
only  when 
requested. 

Feedforward 
advice,  only 
on  request. 

COMMANDED 

Under 

Command 

None 

Full 

Pilot 

None 

performance  is 
transparent. 
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This  is  an  important  step  to  make  the  operational  relationship  between  cooperating  units  explicit.  From  the 
system  engineer’s  point  of  view,  looking  at  the  main  work  system  elements,  the  considerations  following  the 
scheme  as  shown  in  [Taylor,  01]  have  to  be  done  separately  for  the  element  of  operation-assisting  means  and 
the  operator/ACU  team. 

Consequently,  there  are  two  separate  tables  (see  table  2  and  3)  for  the  authority  levels  of  these  two  work 
system  elements.  Only  table  3  contains  the  operational  mode  of  full  computer  authority  in  terms  of  autonomy. 
In  addition,  this  table  shows  that  the  operator/ACU  team  members  are  also  making  use  of  lower  operational 
modes  of  cognitive  automation  in  the  framework  of  their  roles  as  part  of  the  team  (human  team  example: 
cockpit  crew). 


Table  2:  Authority  Levels  and  Corresponding  Operational  Modes  (Operation-Assisting  Means) 


Primary  Modes 

Mode 

Sublevels 

Operational 

Relationship 

Computer 

Authority 

Operator 

Authority 

Adaption 

Information  on 
Performance 

Operator- ACU 

COOPERATIVE 

ASSISTANCE 

(COGNITIVE 

AUTOMATION) 

3 

Direct 

Support 

Action  unless 
revoked 

Revoking 

action 

ACU  backed 
up  by 

Operator 

Feedback  on  action. 
Alerts  and  warnings  on 
failure  of  action. 

2 

In 

Support 

Advice,  and  if 

authorized, 

action 

Acceptance 
of  advice  and 
authorizing 
action 

Operator 
backed  up 
by  the  ACU 

Feed-forward  advice 
and  feedback  on 
action.  Alerts  and 
warnings  on  failure  of 
authorised  action. 

1 

Advisory 

Advice 

Acceptance 
of  advice 

Operator 
assisted  by 
ACU  advice 

Feed-forward  advice 

CALLED 

ASSISTANCE 

(COGNITIVE 

AUTOMATION) 

At  Call 

Advice  and 
action  only  if 
requested 

Full 

Operator 
assisted  by 
ACU  only 
when 
requested. 

Feed-forward  advice, 
only  on  request 
Feedback  on  action. 
Alerts  and  warnings  on 
failure  of  action. 

CONVENTIONAL 

AUTOMATION 

2 

Direct 

Support 

Action  unless 
revoked 

Revoking 

action 

Computer 
backed  up 
by  operator 

Feedback  on  action. 
Alerts  and  warnings  on 
failure  of  action. 

1 

At  Call 

Action 
if  requested 

Full 

Computer 
backed  up 
by  operator 

Feedback  on  action. 
Alerts  and  warnings  on 
failure  of  action. 

NO 

AUTOMATION 

Operator 

only 

None 

Full 

Operator 

Performance  is 
transparent. 
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Table  3:  Authority  Levels  and  Corresponding  Operational  Modes  (Operator/ACU  Team) 


Primary  Modes 

Mode 

Sublevels 

Operational 

Relationship 

Computer 

Authority 

Operator 

Authority 

Adaption 

Information  on 
Performance 

Operator-  ACU 

AUTONOMY 

(COGNITIVE 

COOPERATION) 

Role 

Autonomy 

Full 

Interrupt 

ACUs  monitor¬ 
ed  by  other 
Cognitive  units 
(e.g.  operator) 

On/off 

Failure  warnings. 
Performance  only  if 
required. 

COOPERATIVE 

ASSISTANCE 

(COGNITIVE 

COOPERATION) 

3 

Direct 

Support 

Action  unless 
revoked 

Revoking 

action 

ACU  backed  up 
by  other 
Cognitive  unit 
(e.g.  operator) 

Feedback  on  action. 
Alerts  and  warnings  on 
failure  of  action. 

2 

In 

Support 

Advice,  and 

If  authorized, 
action 

Acceptance 
of  advice  and 
authorizing 
action 

Cognitive  unit 
(e.g.  operator) 
backed  up  by 
the  ACU 

Feed-forward  advice 
and  feedback  on  action. 
Alerts  and  warnings  on 
failure  of  authorised 
action. 

1 

Advisory 

Advice 

Acceptance 
of  advice 

Cognitive  unit 
(e.g.  operator) 
assisted  by 

ACU  advice 

Feed-forward  advice 

CALLED 

ASSISTANCE 

(COGNITIVE 

AUTOMATION) 

At  Call 

Advice  and 
action  only  If 
requested 

Full 

Cognitive  unit 
(e.g.  operator) 
assisted  by 

ACU  only  when 
requested 

Feed-forward  advice, 
only  on  request. 

NO  AUTOMATION 

Operator 

only 

None 

Full 

Operator 

Performance  is 
transparent. 

Table  2  also  has  to  take  into  aecount  the  level  of  Conventional  Automation  as  the  lowest  authority  level  of  a 
primary  mode  using  automation. 

CAMA  is  working  essentially  under  the  same  principals  as  “Cogpit”,  not  going  beyond  the  use  of  ACUs  for 
operation-assisting  means,  i.e.  working  in  correspondence  with  table  3.  At  the  time  being,  this  is  how  far 
computer  authority  is  driven  in  the  flight  domain.  This  will  be  different  in  the  application  domain  of 
unmanned  air  vehicles.  Slightly  different  from  Cogpit,  the  operation  of  CAMA  does  not  include  explicit 
operator  selection  of  authority  changes  during  pre-flight  planning.  The  operator  is  selecting  by  doing.  Pre-set 
action  defaults  are  exclusively  implemented  for  advice  initiatives  subject  to  certain  rules  which  follow  the 
principal  of  being  as  quiet  as  possible.  Emphasis  is  put  on  the  context-dependent  changes  of  automation  level 
as  also  provided  by  PACT  in  the  Cogpit  system,  which  are  either  initiated  by  the  operator  or  by  the  cognitive 
automation  functions.  Figure  7  shows  the  overall  pilot  scoring  of  the  CAMA  flight  tests,  which  somehow 
underline  that  cognitive  automation  in  terms  of  Cooperative  Assistance  brings  about  a  significant  effect. 
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CAMA  increases  Flight  Safety 


Agreed 

irtk 

★ 

Disagreed 

CAMA  increases  Mission  Effectiveness 

Agreed 

irtrk 

★ 

I  Disagreed 

CAMA  increases  Survivability 

Agreed 

irfrk 

★ 

I  Disagreed 

Figure  7:  Overall  Pilot  Scoring  of  the  CAMA  Flight  Tests  (4  Pilots). 

It  turns  out  that  for  the  purpose  of  effeetive  authority  eoordination  in  a  eooperating  team  the  team  members 
should  know  as  much  as  possible  from  each  other’s  performance  characteristics  and  behavioral  traits. 
Therefore,  modelling  of  cognitive  behavior,  in  particular  of  human  behavior,  has  been  used  in  these  systems 
and  has  become  a  major  field  of  further  research.  This  will  be  addressed  briefly  in  the  last  section  of  this  talk. 


4.0  ADAPTIVE  OPERATOR  BEHAVIOR  MODEL 

When  addressing  the  conception  of  ACUs,  the  content  of  the  ‘a  priori’  knowledge  has  not  been  specified  in 
great  detail.  One  aspect,  however,  not  clearly  mentioned  so  far,  should  be  covered  at  this  point  for  further 
appreciation  regarding  cognitive  cooperation.  It  turns  out  that  for  the  purpose  of  effective  authority 
coordination  in  a  cooperating  team  the  team  members  should  know  as  much  as  possible  from  each  other’s 
performance  characteristics  and  behavioral  traits.  Therefore,  modelling  of  cognitive  behavior,  in  particular  of 
human  behavior,  has  been  used  in  these  systems  and  has  become  a  major  field  of  further  research. 
These  models  are  to  be  part  of  the  ‘a  priori’  knowledge  containing  models  of  all  objects  of  the  world 
surrounding  the  ACU  and  being  of  relevance  concerning  the  prime  work  system  objectives.  These  models  are, 
for  instance,  to  be  instantiated  in  the  cognitive  subprocess  of  situation  interpretation. 

Operator  models  have  been  developed  by  use  of  a  wide  range  of  different  paradigms,  methods  and 
applications.  [Jiirgensohn,  97]  gives  a  quite  comprehensive  overview  about  the  actual  state  of  the  art.  The  type 
of  model  to  look  for  is  a  situation-  and  operator-adaptive  model,  i.e.  modelling  the  individual  operator  person, 
actually  working  on  the  work  object. 

In  pursuit  of  a  psychologically  plausible  model,  there  has  been  done  relatively  little  work  in  this  direction. 
A  good  starting  point  is  Rasmussen’s  scheme  of  three  levels  of  human  cognitive  behavior:  the  skill-based, 
the  rule-based  and  the  knowledge-based  level  [Rasmussen,  83].  Behavior  modelling  on  the  rule-based  and  the 
knowledge-based  level  seems  to  be  relatively  straightforward  on  the  basis  of  comprehensive  offline 
interviewing  of  the  operator  person,  if  possible.  On  the  other  hand,  the  probability  of  incompleteness  of  the 
model  with  respect  to  work  situations  not  covered  by  the  interviews  might  still  be  a  problem.  Even  more 
difficult  is  the  modelling  of  skill-based  behavior  on  the  basis  of  verbal  interviews. 
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Therefore,  in  the  following  I  will  address  an  encouraging  approach  for  online  learning  of  the  operator 
behavior,  which  offers  the  chance  to  achieve  the  wanted  situation-  and  operator-adaptation  of  the  model  which 
we  want  for  the  sake  of  effective  cognitive  cooperation.  Amazingly,  there  has  been  done  relatively  little  work 
in  pursuit  of  a  psychologically  plausible  model  of  this  kind.  Yet,  noticeable  progress  of  operator  modelling 
can  be  stated,  for  instance  in  the  domain  of  road  vehicle  driving,  which  has  to  a  large  extent  to  deal  with  the 
allegedly  most  difficult  part  of  behavior  modelling:  Modelling  of  skill-based  behavior.  This  will  briefly  be 
addressed  in  the  following. 

The  learning  process  for  the  driver  model,  the  ACU  could  adopt,  is  a  rather  complex  hybrid  procedure. 
For  each  driving  situation  in  a  particular  traffic  scenario,  a  priori  knowledge  can  be  provided  for  the  rather 
small  number  of  pertinent  candidates  of  action  patterns  [Jensch,  78].  The  decision  process  (see  figure  8) 
for  one  of  these  candidates/altematives  represents  part  of  the  model.  The  driver’s  choice  is  made  according  to 
his  skill-based  situation  interpretation,  including  his  individual  goals,  experiences  and  anticipations  of  the 
behavior  of  the  other  traffic  participants.  This  decision  model  can  be  learned,  for  instance,  by  reasoning  on  the 
basis  of  a  case  base  (see  figure  8). 


Figure  8:  Process  of  Learning  of  a  Driver-Adaptive  Decision  Modei,  Deciding 
on  One  of  the  Situation-Dependent  Action  Pattern  Candidates. 

In  order  to  provide  the  case  base,  the  method  of  process  observation  (on-line  learning)  is  used  based  on  the 
separate  skill-based  action  pattern  model  which  provides  the  actual  “atomic”  driver  action.  This  basic  part  of 
the  driver  model  is  derived  by  a  separate  learning  process,  again  using  the  method  of  process  observation  in 
order  to  teach  the  mapping  from  perception  to  control  actions,  the  stimulus-response  relationship.  In  order  to 
enable  online  learning  the  learning  algorithm  has  to  comply  with  the  following  requirements.  The  algorithm 
must  be  able  to: 

•  operate  with  very  little  a  priori  knowledge 

•  extend  the  knowledge  without  forgetting  that  already  learned 

•  allow  learning  state  monitoring 
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In  order  to  comply  with  these  requirements,  for  instance  general  neural  regression  networks  are  used  for  the 
learning  process.  General  regression  neural  networks  (GRNN)  [Specht,  91]  are  special  RBF  (radial  basis 
function)  networks  for  linear  and  nonlinear  regression  analysis,  which  approximate  an  underlying  probability 
density  distribution  function  using  the  Parzen  window  method  [Parzen,  62]. 

Two  examples  of  results  are  briefly  discussed,  concerning  the  more  fundamental  part  of  learning  the  skill- 
based  driver  action  pattern  [von  Garrel  et  al.,  00].  These  are  examples  for  certain  driving  situations  in 
longitudinal  vehicle  guidance. 

The  first  example  shows  a  representative  result  of  the  behavior  of  a  trained  driver  model  for  the  two 
subsequent  situations  „free  driving”  and  “stopping  at  a  traffic  light”.  A  learning  algorithm  was  used  as 
described  before.  In  the  training  phase  for  the  model,  two  situation  features  (speed,  relative  distance  from 
traffic  light)  are  presented  to  only  one  network  for  the  situation  “stopping  at  traffic  light”.  For  the  situation 
“free  driving”  only  the  feature  “speed”  is  used.  As  shown  in  figure  9,  the  model  for  acceleration  control  drives 
in  a  stable  way  in  the  simulator  test  run  and  behaves  very  similar  to  the  driver. 

a  [m/s''^ 


Figure  9:  Acceleration  Control:  Driver  Model  Performance. 

The  second  example  represents  a  result  for  the  driving  context  of  “car  following”.  The  learning  algorithm  is 
working  on  the  situation  features: 

•  Headway  distance 

•  Relative  speed 

•  Acceleration  of  the  vehicle  in  front 

•  Speed  v[m/s]  of  the  own  vehicle 

Figure  10  shows  the  driver  model  performance  by  comparing  the  actions  for  speed  control  between  driver  and 
model  in  the  situational  context  of  “car  following”  in  a  3  minutes  simulator  test  run  in  an  urban  scenario. 
Again,  the  learning  performance  appears  to  be  amazingly  well.  The  discrepancies  between  driver  and  model 
are  rather  small,  mostly  well  within  the  scatter  which  is  shown  by  the  driver  anyway. 
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Speed  [m/s] 


Figure  10:  Simulator  Test  Run  with  Speed  Control  in  the  Situational  Context  of  “Car  Following”: 
Driver  Model  Performance,  Comparison  Between  Model  and  Driver  Behavior. 


5.0  CONCLUDING  REMARKS 

The  advances  in  cognitive  engineering  technology  have  brought  about  means  to  systematically  reflect 
requirements  of  human-centered  design  into  clear-cut  work  system  specifications.  In  particular, 
the  improvement  of  the  operation-assisting  means  as  part  of  the  work  system  is  emphasized  on  the  basis  of 
cognitive  automation,  a  new  higher  level  automation  capability. 

Prototype  systems  have  already  shown  -  and  there  is  prospect  of  further  considerable  extensions  and 
performance  improvement  -  that  cognitive  automation  can  indeed  provide  highly  effective  cognitive 
cooperation  for  the  sake  of  work  process  productivity  and  work  satisfaction  of  the  human  operator. 

The  success  of  cognitive  cooperation  in  the  work  process  is  highly  dependent  on  the  capability  of  the  artificial 
cognitive  units  involved  to  found  its  activities  on  a  valid  model  of  the  human  operator.  Developments  in  this 
area  for  situation-  and  operator-adaptive  models  have,  by  now,  arrived  at  a  very  encouraging  performance 
level. 

What  is  not  addressed  but  should  be  mentioned  at  this  point:  The  cognitive  system  approach  is  also  a  very 
powerful  means  to  evaluate  the  work  system  performance  on  an  objective  basis,  also  for  networks  of 
interdependent  work  sites.  This  will  help  to  reveal  and  clarify  where  the  weaknesses  lie  of  existing  work 
systems  and  those  in  development. 
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